Planning and Learning with Adaptive Lookahead

Some of the most powerful reinforcement learning frameworks use planning for action selection. Interestingly, their horizon is either fixed or determined arbitrarily by state visitation history. Here, we expand beyond naive and propose a theoretically justified strategy adaptive selection as function state-dependent value estimate. We two variants lookahead analyze trade-off between iteration count computational complexity per iteration. then devise corresponding deep Q-network algorithm with an tree search horizon. separate estimation depth to compensate off-policy discrepancy depths. Lastly, demonstrate efficacy our method in maze environment Atari.